Computer vision based interfaces for computer games

Interacting with a computer game using only a simple web camera has seen a great deal of success in the computer games industry, as demonstrated by the numerous computer vision based games available for the Sony PlayStation 2 and PlayStation 3 game consoles. Computational efficiency is important for these human computer interaction applications, so for simple interactions a fast background subtraction approach is used that incorporates a new local descriptor which uses a novel temporal coding scheme that is much more robust to noise than the standard formulations. Results are presented that demonstrate the effect of using this method for code label stability. Detecting local image changes is sufficient for basic interactions, but exploiting high-level information about the player’s actions, such as detecting the location of the player’s head, the player’s body, or ideally the player’s pose, could be used as a cue to provide more complex interactions. Following an object detection approach to this problem, a combined detection and segmentation approach is explored that uses a face detection algorithm to initialise simple shape priors to demonstrate that good real-time performance can be achieved for face texture segmentation. Ultimately, knowing the player’s pose solves many of the problems encountered by simple local image feature based methods, but is a difficult and non-trivial problem. A detection approach is also taken to pose estimation: first as a binary class problem for human detection, and then as a multi-class problem for combined localisation and pose detection. For human detection, a novel formulation of the standard chamfer matching algorithm as an SVM classifier is proposed that allows shape template weights to be learnt automatically. This allows templates to be learnt directly from training data even in the presence of background and without the need to pre-process the images to extract their silhouettes. Good results are achieved when compared to a state of the art human detection classifier. For combined pose detection and localisation, a novel and scalable method of exploiting the edge distribution in aligned training images is presented to select the most potentially discriminative locations for local descriptors that allows a much higher space of descriptor configurations to be utilised efficiently. Results are presented that show competitive performance when compared to other combined localisation and pose detection methods. Dedicated to my parents and my family, to family Dahm, and to Susanne. In memory of Elke Dahm, and Winnie Smith.